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MODELING  THE 
PERFORMANCE  AND 
RISKS  OF 
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ACQUISITION 


\  David  N.  Ford  and  COL  Jd[ 
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FROM  AMORPHOUS 
TO  DEFINED: 
BALANCING  RISKS 
IN  EVOLUTIONARY 
ACQUISITION 

^  COL  John  T.  Diitard ,  US.A  (Bet)  and  David  N.  Ford 

Incremental  development  entails  the  deliberate  deferral  of 
worn  to  a  subsequent  period,  using  technology  maturity  as  the 
measure  of  readiness.  This  a  rticle  illustrates  that  this  approach 
might  enable  more  effective  delivery  of  the  first  Increment 
With  a  comparison  of  two  major  systems  is  cue  studies.  But 
there  are  some  Inherent  risks  In  an  evolutionary  approach 
as  wall,  and  the  authors  cau  trait  that  curtain  attnbLj&Bt  of 

hardware  products  might  help  determine  the  suitability  of 

ovolubcinary  deWoprntint  methodologies,  Mutable  prod¬ 
ucts  with  costless  production,  continuous  requirements,  tow 
maintenance,  or  time  criticality  may  be  more  likely  to  reap 
advantages  from  evolutionary  approaches.  Products  that  are 
nearly  Irtimulabl  dl  have  binary  requirements  Far  key  capabili¬ 
ties,  require  man-rating,  or  are  maintenance-intensive  may 

not  ba  best  candidates  for  incrnmont.il  dovuiopmant. 
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Overview  of  Presentation 

•  DoD’s  evolving  policy  on  acquisition  management 

-  How  our  systems  management  model  has  changed 

•  Previous  Research  on  Life-Cycle  Models 

-  How  many  “control  gates”  do  you  need? 

•  Evolutionary  Acquisition  and  its  implications 

-  Case  studies 

-  Organizational  and  System  Dynamics  modeling 

•  Recent  views  of  “the  System”  and  how  it  is  today 
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Different  Approaches  &  Terminology 


•  Progressive  elaboration  (vs.  “Requirements  creep”) 

•  Iterative  design/rapid  prototyping 

•  Pre-planned  product  improvement 

-  Evolutionary  acquisition 
-  Spiral  development 
-  Incremental  capability 
-  Planned  upgrades 
-  Rational  Unified  Process  Framework 

VERSUS: 

•  Single  Step 

•  Grand  design 

*  Unified  Development  Method 
•  Technological  leap 
*  Waterfall 


Development  Life-Cycle  Models 


“The  best  material 
model  of  a  cat  is 
another ;  or  preferably 
the  same,  cat  ” 


Norbert  Wiener,  1948 


“All  models  are  wrong. 
Some  are  useful.” 


George  E.  P.  Box,  1979 


The  Defense  Acquisition  Management  System 

as  of  December  2008 


User  Needs 


Technology  Opportunities  &  Resources 


A  A 


A 


IOC 


FOC 


Materiel 

Solution 

Analysis 

Technology 

Development 

Engineering  and 
Manufacturing 

Production  &  Deployment 

Operations  & 
SuDDort 

Development 

JjMateriel 

^Development 

1  Decision 

yPostCDR 

A  FRP 

/  Decision 

*  Assessment 

f 

Review 

Pre-Systems 

Acquisition 

Systems  Acquisition  ^ 

1  Sustainment 

This  general  graphic  has  served  for  decades  as  DoD’s 
“Life-Cycle  Systems  Management  Model” 
as  well  as  it’s  decision  “framework.” 
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DoD  Leadership’s  Stated  Intent 
for  DoD  5000  Revision 

“....create  an  acquisition  policy 
environment  that  fosters  efficiency, 
flexibility  creativity  and  Innovation.  ” 

DEPSECDEF  Wolfowitz,  30  Oct  2002 

Revised  Policy  Objective: 

•  Encourage  innovation  and  flexibility 

•  Decentralized  responsibility  to  be  maximized 

•  Empower  PM’s  to  use  the  system  vice  being 
hampered  by  over-regulation 

•  Minimize  reporting  requirements 


MEMORANDUM  FOR  SECRETARIES  OF  THE  MILITARY  DEPARTMENTS 
CHAIRMAN  OF  THE  JOINT  CHIEFS  OF  STAFF 
UNDER  SECRETARIES  OF  DEFENSE,  et  al. 

OCT  30  2002 

SUBJECT:  Defense  Acquisition 

I  have  determined  that  the  current  DoD  Directive  5000.1 ,  ‘The 
Defense  Acquisition  System,"  DoD  Instruction  5000.2,  “The 
Operation  of  the  Defense  Acquisition  System,”  and  DoD  5000.2 -R, 
Mandatory  Procedures  for  Major  Defense  Acquisition  Programs 
(MDAPs)  and  Major  Automated  Information  System  (MAIS) 
Acquisition  Programs,”  require  revision  to  create  an  acquisition 
policy  environment  that  fosters  efficiency,  flexibility,  creativity,  and 
innovation.  Therefore,  by  separate  memorandum.  I  have  cancelled 
those  documents  effective  immediately 


By  this  memorandum,  I  am  issuing  the  attached  interim  guidance  in 
place  of  the  cancelled  documents.  The  intent  of  the  guidance  is  to 
rapidly  deliver  affordable,  sustainable  capability  to  the  warfighter  that 
meets  the  warfighter's  needs.  Additional,  supporting  discretionary,  best 
practices,  lessons  learned,  and  expectations  have  been  posted  to  the  DoD 
5000  Resource  Center  at  http://dod5000.dau  .mil. 


I  am  directing  the  Under  Secretary  of  Defense  for  Acquisition, 
Technology,  and  Logistics,  with  the  Assistant  Secretary  of  Defense 
(Command,  Control,  Communications,  and  Intelligence)  andtho  Director, 
Operational  Test  and  Evaluation,  to  jointly  prepare  revised  documents 
within  120  days. 
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CD  A 


C 


B 


DRR 


FRPDR 


DoDI  5000.2  of  May  2003 


Initial  Research  Methodology 


*  Turbulence  in  policy  &  confusion  in  the  field 

*  Complexity  of  the  new  model: 

-  More  decision  reviews 

-  Higher  level  of  reviews 

-  Placement  of  reviews  and  project  events 


1 .  What  other  Project  Management  models  exist? 

2.  (Explicit  and)  Implicit  aspects  of  the  new  model? 

3.  Congruent  with  stated  policy? 

4.  Best  fit  to  environment  vis-a-vis  Organizational  theory? 
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Project  Management  Institute’s 
Generic  Project  Models 


Key  Tenets 
of  Projects: 

—  Concurrency 


—  Phasing 
"Control  Gates" 
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Construction  Industry  Project  Model 
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Software  Industry  Project  Model 
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Pharmaceutical  Industry  Project  Model 
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Discovery 

Preclinical 

Screening  Development 

Reqistration(s)  Workup 

Postsubmission  Activity 

- ►  Ten  Plus  Years  — 

- ► 

Most  similar  to  DoD?: 

•  Serial  vs.  Concurrent  Orientation 

•  Prmary  Metrics:  Safety  &  Efficacy 

•  Average  Project:  $897M  &  10  yrs 

•  Both  Govt  and  Private  funded  R&D 

•  Lengthy  FDA  Review 


True  Comparison  of  1996  and  2003  Models 

Under  an  Evolutionary  Acquisition  Strategy 
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Typical  Program: 

16  OSD-Level  Reviews  in  12  Years 
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(PM  Toolkit,  2009)  i6 
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(PM  Toolkit,  2009) 


ADM  -Acquisition  Decision Memorandum 
CARD  -  Cost  Analysis  Requirements  Description 
FRPDR  -  Full  Rate  Production  Decision  Review 
JROC  -  Joint  Requirements  Oversight  Council 


CAI$  -  Cost  Analysis  Improvements  $roup 
DAD  -  Defense  Acquisition  Do  and 
JCD-  Joint  Capabilities  Doarct 
LCCE  -  Life  cycle  cost  estimates) 
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Defense  Acquisition  Review  Journal 

December  2005 
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TOWARD  CENTRALIZED 
CONTROL  OF  DEFENSE 
ACQUISITION  PROGRAMS 

John  T.  Dillard 


A  great  deal  of  lurbulence  in  U.S.  defense  acquisition  policy  has  contributed 
to  confusion  during  the  last  three  years  within  the  acquisition  workforce  in 
terminology,  major  policy  thrusts,  and  unclear  implications  of  the  changes. 
The  new  acquisition  framework  has  added  complexity  with  more  phases  and 
delineations  of  activity,  and  both  the  number  and  level  of  decision  reviews  have 
increased.  As  a  result,  program  managers  may  now  have  fewer  resources  to 
manage  their  programs  as  they  spend  much  of  their  time  and  budgets  managing 
the  bureaucracy.  This  same  framework  and  its  associated  requirements  for  senior 
level  reviews  are  opposed  to  the  rapid  and  evolutionary  policy  espoused  and 
are  counter  to  appropriate  management  strategies  for  a  transformational  era. 


The  issuance  of  Department  of  Defense  (DoD)  Directive  5000.1  (2003)  and  DoD 
Instruction  5000.2  (2003)  is  the  third  significant  revision  of  acquisition  policy  in 
many  years.  Looking  further  back,  these  three  revisions  of  regulatory  guidance 
evolved  from  two  previous  versions  in  1991  and  1996.  Each  had  its  major  thrusts 
and  tenets,  and  perhaps  of  most  importance  to  program  managers:  each  modified  the 
“Defense  Systems  Acquisition  Management  Process"'  (Defense  Systems  Management 
College  [DSMC],  2001)  or  “Defense  Acquisition  Framework”  (DSMC.  2001).  which 
is  the  broad  paradigm  of  phases  and  milestone  reviews  in  the  life  of  an  acquisition 
program.  The  purpose  of  this  research  was  to  examine  the  evolution  of  this  framework 
and  explain  the  explicit  and  implicit  aspects  of  recent  changes  to  the  model  to  better 
understand  its  current  form.  Provided  here  is  a  synopsis  of  the  most  important  findings. 
The  full  report  of  this  research,  examining  both  private  industry  and  defense  acquisition 
decision  models  is  available  for  a  more  in-depth  review  (Dillard,  2003). 

The  very  latest  DoD  5000  policy  changes  came  during  a  time  of  DoD  transformation, 
which  is  chiefly  focused  on  changes  to  force  structure  and  weapons  employment 
capabilities.  The  latest  version  of  the  5000  series  was  actually  drafted  in  the  documents 
rescinding  its  predecessor.  According  to  a  memorandum  signed  by  Deputy  Secretary 

331 
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Contingency  Theory 

•  Organizational  structures  must  change  in 
response  to  contingencies  of  size,  technology, 
and  as  external  environments  become  more 
complex  and  dynamic. 

•  Command  structure  must: 

-  (a)  demand  more  information,  or 

-  (b)  enable  local  forces  to  deal  with  the  situation. 

•  Research  supports  decentralized  control  as 
preferred  approach. 

(see  Galbraith,  1973  and  Van  Creveld’s  Command  in  War,  1985) 


19 


Citations  Elsewhere 


Report  of  the 

Defense  Science  Board  Task  Force 

on 

Management  Oversight  in  Acquisition 
Organizations 


March  2005 


O  ffice  of  the  Under  Secretary  of  Defense 
For  Acquisition,  Technology,  and  Logistics 
Washington,  D.C.  20301-3140 


Beyond  Goldwater-Nichols : 

U.S.  Government  and  Defense  Reform 
for  a  New  Strategic  Era 

Phase  2  Report 


Lead  Investigators 

Clark  A.  Murdock 
Michele  A.  Flournoy 

Principal  Authors 

Clark  A.  Murdock 
Michele  A.  Flournoy 
Kurt  M.  Campbell 
Pierre  A.  Chao 
Juliannc  Smith 
Anne  A.  Witkowsky 
Christine  E.  Womnuth 

Contributors 

Mac  Boll  man 
Jeremiah  Gertler 
Adam  N.  Marks 
Noah  J.  Richmond 
David  R.  Scruggs 
Richard  Weitz 


DEFENSE  ACQUISITION 

PERFORMANCE  ASSESSMENT 

EPOR 


Foreword  by  Norman  Augustine 

,c 


January  2006 


July  2005 


CSS 


20 


Affirmations  Continue 


jNPSj 

k  ^RMST^TlAPERsCIfNnAA1  , 

vaagar 


“We’re  in  an  endless  cycle  of  reviews” 


“One  review  per  year  is  pretty  much  the 
norm  now...  There’s  things  I’d  rather  be 
doing. ..I  just  appointed  a  colonel  to  a  new 


position  -  to  look  across  my  programs” 


Harvard  Business  Review 


* 


September  2003 
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Why  Good  Projects  Fail  Anyway 

September  2003 

Challenging  senior  leaders  to  cede  control: 

“Managers  expect  they  will  be  able  to  identify,  plan  for,  and  influence  all 
the  variables  and  players  in  advance,  but  they  can’t.  Nobody  is  that  smart 
or  has  a  crystal  ball.  They  can,  however,  create  an  ongoing  process  of 
learning  and  discovery,  challenging  the  people  close  to  the  action  to 
produce  results  -  “ 


Nadim  F.  Matta  &  Ronald  Ashkenus 
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Evolutionary  Development  as  Mandate 


“Evolutionary  acquisition  strategies  shall  be  the 
preferred  approach  to  satisfying  operational  needs.” 

DoDI  5000.2 


Evolutionary  Acquisition  Model 


Feedback  from  both  technology  development  and 
from  the  testers  &  users  help  to  define  future  product 
iterations. 


Key 


i 

Technology  1 

^User 

Feedback 

Feedback 

Time 


Evolutionary  Acquisition 


Increment  2 


Further  defined: 

•  l_ ncremenUd  Development :  A  desired  capability  is  identified;  the 
end-state  requirement  is  known :  and  that  requirement  is  met  over 
time  by  developing  several  increments,  each  dependent  on 
available,  mature  technology. 

•  Spiral  Development :  A  desired  capability  is  identified,  but  the  end- 
state  requirements  are  not  known  at  program  initiation. 
Requirements  are  refined  through  demonstration  and  risk 
management;  there  is  continuous  user  feedback;  and  each 
increment  provides  the  user  the  best  possible  capability. 


Development  Strategy  Comparison  Table 


^\4>trategy  or  Development 

Process 

Criteria 

Single  Step 
to  Full 
Capability 

Pre-planned  Product 
Improvement  (P3i) 

Evolutionary 

Acquisition 

Incremental 

Development 

Spiral 

Development 

Full  requirements 
defined  at  outset 

Yes 

Yes 

Yes 

No 

Useful  intermediate 
capabilities 

No 

Yes 

Yes 

Yes 

Multiple  iterations 

No 

No 

Yes 

Yes 

All  capabilities 
required  in  initial 
increment 

Yes 

No 

No 

No 

User  feedback  from 
earlier  iterations 
used  to  define  final 
requirement 

No 

No 

Yes 

Yes 

Other  characteristics 

Used  as  the 
traditional 
acquisition 
strategy 

Achieves  increased 
capability  from 
maturing  technology 
with  architecture  in 
place 

Developmental 
process  when  full 
requirements  defined 
at  outset 

Developmental 
process  when  full 
requirements  not 
defined  at  outset 

*  * 
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United  States  Code 

TITLE  10,  Subtitle  A,  PART  IV,  CHAPTER  144,  §  2430 


“(g)  Definitions. — In  this  section: 

“(1)  The  term  ‘spiral  development  program’,  with 
respect  to  a  research  and  development  program, 
means  a  program  that — 

“(A)  is  conducted  in  discrete  phases  or  blocks,  each  of 
which  will  result  in  the  development  of  fieldable 
prototypes;  and 

“(B)  will  not  proceed  into  acquisition  until  specific 
performance  parameters,  including  measurable  exit 
criteria,  have  been  met. 


F-18  E/F  Super  Hornet 
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Evolutionary  Development  in  Practice 


Paradigm 


Paradigm 


“Paradigms  nfluence 
our  study  between 
revolutions”  Kuhn 
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Markets  Love  Product  Variety 
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Product  Variety  Has  Downsides 


Supply,  Maintenance  and  Training  Impacts 
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Unwanted  Variety  in  Production 


Failures  and  Failure  Mode  Identification  28 


CSIS  Study  Panel  on  Spiral  Development 


HOME 

Center  for  Strategic  &  International  Studies 

PROVIDING  STRATEGIC  INSIGHTS  AND  POLICY  SOLUTIONS 

EVENTS 

ABOUT  CSIS  EXPERTS  RESEARCH  FOCUS  CSIS  &  CONGRESS  PUBLICATIONS  PRESS  CENTER  EVENTS 


SEARCH 


Home  page  »  Euents  »  Spiral  Deuelopment,  Real  Options,  and  Other  Deuelopment  Methodologies 


OPEN  EVENT  DETAIL 

SPIRAL  DEVELOPMENT,  REAL  OPTIONS,  AND  OTHER  DEVELOPMENT 
METHODOLOGIES 


DATE:  June  5,  2006 

TIME:  3:30  a.m. -12:1 5  p.m. 

LOCATION:  CSIS 

B-1  Conference  Center 
1 80O  K  Street,  N.W. 
Washington,  D.C. 


is 
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Integrated  vs.  Serial  Approach 


Development  &  Testing  Support 


“ Intelligent  design  is  way  faster  than  evolution 

Robert  N.  Metcalfe 


Research  on  Evolutionary  Acquisition 


United  States  Government  Accountability  Office 

GAO 

Report  to  Congressional  Committees 

March  2007 

DEFENSE 

ACQUISITIONS 

RAND 


PROJECT  AIR  FORCE 


RESEARCH 

BRIEF 


"Evolutionary  Acquisition"  Is  a  Promising 
Strategy,  But  Has  Been  Difficult 
to  Implement 


RAND  RESEARCH  AREAS 
THE  ACTS 
CHIID  POLICY 
aviL  JUSTICE 
EDUCATION 
ENERGY  AND  ENVIRONMENT 
HEALTH  AMJ  HEALTH  CARE 
INTERNATIONAL  AFFAJRS 
NATIONAL  30JBITY 
POPULATION  AM}  AGING 
PLJBUC  SAFETY 
SCIENCE  AM}  TECHNOLOGY 


In  2003,  the  U.S.  Depart  men  L  of  Defense  (DoD.J  specified  evolutionary  acquis  it  ion  (EA)  as  the 
preferred  approach  to  wcapoLi  system  acquisition,  a  Lid  spiral  development  as  the  preferred  means  of 
Implementation.  EA  strategics  aim  to  develop  new  capabilities  in  multiple  increments,  as  opposed  to 
the  traditional  strategy  of  developing  a  full  capability  in  a  single,  lengthier  step.  EA  strategies  are  meant 
to  reduce  the  rime  it  rakes  to  field  operationally  useful  equipment,  control  technical  risk  ajid  cost  growth, 
and  make  cost  estimates  more  reliable  I ■  ir  each  stage  ol  development,  while  allowing  greater  flexibility 
to  evaluate  and  improve  a  program  based  on  experience  in  the  field.  "Hi Ls  greater  flexibility  arises  in  pari 
horn  the  fact  that,  with  the  spiral  development  approach,  the  end-state  requirements  are  noi  known  at 
program  initial  Lon,  but  rather  emerge  and  evolve  through  an  iterative  process  ol  phased  development  and 
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A  Tale  of  Two  Missiles 


Spiral  and  Incremental 
Development 

ARMY  TACMS 

MISSILE  DESIGNED  FOR  GROWTH  WARHEADS 

M74  Submunition 


Single  Step  to  Full 
Capability 
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Technology  Readiness 
Levels 

Level 

Hardware 

(and  software  necessary  to 
demonstrate  capability) 

Environment 

1  -  Basic  principles  observed  and 
reported 

Studies 

None 

None 

2  -  Technology  concept  and/or 
application  formulated 

Studies 

None 

None 

3  -  Analytical  and  experimental 
critical  function  and/or 
characteristic  proof  of  concept 

Component 

Nonscale  components 
(pieces  of  subsystem) 

Lab 

4  -  Component  and/or  breadboard 
validation  in  lab  environment. 

Component/ 

subsystem 

Low  fidelity  breadboard 

(integration  of  nonscale  components 
not  fully  functional  or  form  and  fit) 

Lab 

5  -  Component  and/or  breadboard 
validation  in  relevant  environment 

Subsystem 

High  fidelity  breadboard 

(functionally  equivalent  but  not  form 
and  fit) 

Lab  or  may  include  flight 
demo  in  surrogate  aircraft 

6  -  System/subsystem  model  or 
prototype  demonstration  in  relevant 
environment 

Subsystem 

Prototype 

(should  be  very  close  to  form,  fit  and 
function) 

Lab  or  limited  flight 
demonstration 

7  -  System  prototype  demonstration 
in  an  operational  environment 

Subsystem 

Prototype 

(form,  fit  and  function) 

Flight  demo  in 
representative  environment 
such  as  test  bed 

8  -  Actual  system  completed  and 
flight  “qualified”  through  test  and 
demonstration 

System 

Flight  qualified  hardware 

DT&E  in  actual  system 
application 

9  -  Actual  system  “flight  proven” 
through  successful  mission 
operations 

System 

Actual  system  in  final  form 

OT&E  in  operational 
mission  conditions 

Technology  Maturity  -  A  Key  Difference 


Key  Program  Characteristics  -  First  Increment  of  Capability 

Program  Aspects 

ATACMS 

JAVLIN 

DARPA  Predecessor 

Assault  Breaker  1977-62 

Tank  Breaker  1901-02 

Ultimate  Capability 

'Deep  Afte*" 

■ffleSflwgef 

Critical  Tochnolooloa  &  Readlneai  Laveli: 

Munition 

9- 

Lance  M74  Bomblet 

5-  Tandem  Shaped  Charges 

Propulilon 

9- 

Solid  Rocket  Motor 

5  -  Two-Stsge  Solid  Rocket  Motor 

Flight  Control 

9- 

Fin  surfaces 

6  ■  Fins  +  TTirust  Vector  Control  Vanes 

Guidance  and  Control 

9- 

Inertial 

4  -  Tracker  Software  Algorithm 

Safb/Aim  Fusing 

7- 

Mechanical 

4- Electronic 

Software  Function  (tusahmih fh oasm 

6- 

Varioua 

6- Various 

Sensor 

N/A 

6  -  Focal  Plane  Array 

Capability  Leap  Area 

Rang* 

Range,  Lethality,  Survivability 

Cost  of  development 

~$700M 

~$700M 

Contract  Type 

Fbcad  Prlea 

Cost  Reimbursable 

Tech  Development  Phase 

0  Months 

27  Months 

Advanced  Development  Phase  -  Planned 

46  Months 

36  Months 

Advanced  Development  Phase  -  Actual 

61  Months 

64  Months 

Total  Time  In  Development 

51  Months 

81  Months 

Advanced  Development  Phase  Contract  Cost  Growth  0% 

>150% 
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Technology  “Push  or  Pull”  ? 


Technology 

Development 

Advanced 

Development 

Initial 

Production 

To  this: 

Technology 

Advanced 

Initial 

Development 

Development 

Production 

35 


Relative  Concurrency  of  Increments 

And  Concomitant  Organizational  Impacts 


Development  Increments  Concurrent  with  Initial  Production 


Development  Increments  Concurrent  with  Later  Production 
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Evolutionary  Acquisition  Issues 

*  Continued  conceptual  and  definitional  ambiguity  (RAND) 

*  Number  of  OSD-Level  Reviews 

-  Off-Core  Activities 

-  Significant  Transaction  Costs 

*  Unplanned  work  (spiral)  is  inestimable 

*  Fielding  of  obsolete  technology  -  if  EMD  isn’t  shortened 

*  1st  Increment  Focus:  All  desired  capabilities  vs.  “Militarily  useful” 

*  Organizational  impacts  of  concurrent  production  and  development 
of  follow-on  increments 

*  Maintaining  of  funding  priority  for  follow-on  increments 

*  GAO  examples  are  mostly  from  cyclical  commercial  models,  versus 
fleet  ownership  (i.e.,  United,  UPS,  Fedex) 

*  Variety  brings  benefits  and  costs 
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Everything  Changes,  But... 


Adobe  Reader  7.0.9  for 
Windows  2000  SP2  -  SP3, 
English 

Latest  version 


A  one-size-fits-all  development 
methodology  may  not  be  appropriate  for  all 

product  commodities. 
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Product  Attributes  May  Affect  the 
Development  Strategy 

•  Production  Quantity  is  not  a  factor 

•  Mutability 

•  Range  of  Requirement  Attainment  (Binary  vs.  Continuous) 

•  User  Risk  (Safety  and  Time  Criticality) 

-  Time-critical  or  enhanced  survivability  systems  (NMD, 
ARCI) 

-  Non-man-rated  Systems  (UAVs) 

-  Man-rated  Systems  (munitions) 

•  Logistical  Support  Planned  During  Service/Shelf  Life 

•  Net  Amount  of  Change  -  and  the  Lure  of  Modularity 

-  Changes  propagate  with  relative  modular  interdependency 
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System  Dynamics:  Work  Flows  and 
Backlogs  through  a  Development  Phase 


Work  flows  are  constrained  by  resources  and 

availability  of  work 
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L^Es&rr  Information  Flows  in  a  Single-block 

[7raest*ntia  msamuAM~f/  w 


Acquisition  Project 


Time  Periods 

1  2  3  4  5  6  7  8  9  10  11  12  13  14  15  16  17  18  19  20  21 


Develop  Requirements 


Develop  Technologies 


Advanced  Development 


Manufacturing 


User  Product  Testing 


Milestones 


Modeling  inter-phase  concurrence  &  information 

dependencies 
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Information  Flows  in  an  Incremental 

Acquisition  Project 


Time  Periods 


Develop  Requirements 


Develop  Technologies 


Advanced  Developmen 


Manufacturing 


User  Product  Testing 

Milestones,  lter#1 
Milestones,  Iter  #2 
Milestones,  Iter  #3 


8  9  10  11  12  13  14  15  16  17  18  19  20  21  22  23  24  25  26  27  28  29  30 


»  Reveals  more  concurrence  and  interdependency 

»  Contracting,  etc.  modeled  with  indirect  work  at  start  of 
each  phase 

•  Reviews  modeled  with  indirect  work  at  end  of  each  phase 
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Impacts  of  Multiple  Development  Blocks 


Units  of 
Measure 

Pr« 

Javelin 
(single  block) 

oject  Scenar 

Base  Case 
(single  block) 

io 

Base  Case (3 
blocks) 

Best 

Performance 

Duration  to  first 
requirement  satisfied 

weeks 

471 

470 

(5) 

Base  Case 
(3  blocks) 

L. 

^  Duration  to  max. 

^  requirements  satisfied 

weeks 

520 

518 

762 

Base  Case 
(single  block) 

Q) 

u  Total  development  cost 
re 

$1,000,000 

$722 

$719 

($1^555) 

Base  Case 
(single  block) 

E 

O  Requirements  satisfied 
^  by  deadline 

<D 

%of 

requirements 

developed 

100 

91 

18 

Javelin  (single 
block) 

Final  requirements 
satisfied 

%of 

requirements 

developed 

100 

91 

91 

Javelin  (single 
block) 

The  (dis)advantages  of  Evolutionary  Acquisition 
depend  on  what  performance  measures  are  most 

important 
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Conclusions:  Evolutionary  vs.  Single 
Block  Development  Approaches... 


•  First  Unit  Equipped  with  some  (but  not  all) 
requirements  satisfied  faster 

•  Requires  more  time  to  satisfy  all  requirements 

•  Costs  more  than  single-block  development  for 


same  requirements 

Higher  risk  of  not  satisfying  all  requirements  by 
the  time  single-block  development  could  do  so 
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Our  Bottom  Line  on  Risks 


•  DoD  uniquely  outsources  development  for  internal  use 
-  but  owns  the  product  over  its  entire  life  cycle 

•  There  are  inherent  potential  risks  with  incremental 
development 

-  inefficiencies  from  re-work  (dupl  cation) 

-  risk  of  project  error  (from  discontinuous  membersh  p) 

-  organizational  impacts  (queuing  theory) 

-  relative  concurrency  drives  risk 

-  variety  in  the  fleet  (support,  failure  mode,  training,  etc.) 

•  Defer  what  you  cannot  do  now  -  tech  readiness 

•  Don’t  defer  what  you  can  do  now 

•  Product  attributes  may  affect  development  strategy 
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Our  Top  Line  on  Control 


•  Rigorous  Preliminary  Effort  on  Architecture 

•  Meticulous  Configuration  Management 

•  Individual  Accountability 

•  Other  control  measures  to  balance  risks 


-  Testing,  Interface  Control,  Peer  Review 

-  Open  Architecture  Incentives,  etc. 
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Agile  vs.  Plan-Driven  Software  Development 


Barry  Boehm 
Richard  Turner 


Balancing  Agility 
and  Discipline 


A  Guide  for  the  Perplexed 

Forewords  by 

Grady  Booch  -  Alistair  Cock burn  -  Arthur  Pysier 

What  Does 
Your 
Project 
Process 
Model  Look 
Like? 


Size 


(Numbw  of  personnel) 


(%  Thriving  m  order} 


Personnel 

(%UjY$nB)  (%Levet2an<}$) 


Criticality 

due  &  tmpa&  of  defect) 


Dynamism 

Pequiremer!te-<Jteinge/mQn<f>) 


L  m> 

Life 


Eunds 
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jNPSi 

k  ^RMST^TlAPERsCIfNnAA1  , 

vaagar 


The  Defense  Acquisition  Management  System 


User  Needs 


Technology  Opportunities  &  Resources 


ICD 


Materiel 

Solution 

Analysis 


Materiel 
►  Development 
Decision 

AoA 


A 


A 


Technology 

Development 


Engineering  and 
Manufacturing  Development 


CDD 


PDR 


/  /\  Post  PDR  /\Post  CDR 
\/ Assessment Assessment 

PDR  CDR 


CPD 


IOC 


Production  & 
Deployment 


FRP 


0r 

Decision 

Review 


Pre-Systems  Acquisition 


Systems  Acquisition 


FOC 


Operations  & 
Support 


Sustainment 


ICD:  Initial  Capabilities  Document 
AoA:  Analysis  of  Alternatives 
PDR:  Preliminary  Design  Review 
CDD:  Capability  Development  Document 
CDR:  Critical  Design  Review 


CPD:  Capability  Production  Document 
FRP:  Full  Rate  Production 
IOC:  Initial  Operational  Capability 
FOC:  Full  Operational  Capability 
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Today’s  Evolutionary  Approach 


DoD 

Strategic  Guidance 


Joint  Operating  Concepts 
Joint  Functional  Concepts 


'55 

<o 

C 

< 

Q. 

<0 

O 


DAB 


DAB 


DAB 


1C 


JROC 


ateriel  Solution 
D^>  Analysis 

Analysis  of 
Alternatives 


Technology 
evelopment 


CDD#  B 


EMD 

CPD1 

k  Increment  1 

i 

JROC 


DAB 


JROC 


DAB  | 

Technology 

Development 

CDD2 

/esl  emd 

- ^  Increment  2 

CPD2 

DAB 


A 


JROC 


JROC 


DAB 


DAB 


Technology 

^Development 


CDD3 


DAB 


T 


EMD 
Increment  3 


CPD3 


JROC 


JROC 


Continuous  Technology  Development  and  Maturation 


MDD:  Materiel  Development  Decision 
JROC:  Joint  Requirements  Oversight  Council 


EMD:  Engineering  &  Manufacturing  Development 
DAB:  Defense  Acquisition  Board 
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Questions/Comments? 


Acronym  Listing 


AFRB  -  Air  Force  Review  Board 

AFROCC  -  Air  Force  Requirements  for  Operational 
Capability  Council 

AFSB  -  Air  Force  Studies  Board 

AOTR  -  assessment  of  operational  test  readiness 

ASP  -  acquisition  strategy  panel 

ASR  -  alternative  system  review 

CD  -  concept  decision 

CDR  -  critical  design  review 

CSB  -  configuration  steering  board 

DAB  -  Defense  Acquisition  Board 

DRR  -  design  readiness  review 

DSAB  -  Defense  Space  Acquisition  Board 

FCA  -  functional  configuration  audit 

FRP  -  full  rate  production 

GAO  -  Government  Accountability  Office 

IBR  -  integrated  baseline  review 

HPT  -  integrating  integrated  product  team 

IPA  -  independent  program  assessment 

IPT  -  integrated  product  team 

JAT  -  joint  assessment  team 


JCIDS  -  Joint  Capabilities  Integration  and 
Development  System 

JROC  -  Joint  Requirements  Oversight  Council 
LHA  -  logistics  health  assessment 
MRA  -  manufacturing  readiness  assessment 
MS  -  milestone 

OIPT  -  overarching  integrated  product  team 

OTRR  -  operational  test  readiness  review 

PCA  -  physical  configuration  audit 

PCDRA  -  post-CDR  assessment 

PDR  -  preliminary  design  review 

PEO/SR  -  program  executive  officer  sufficiency 

review 

PM  -  program  manager 

PRR  -  production  readiness  review 

PSR  -  program  support  review 

SDR  -  system  design  review 

SEAM  -  systems  engineering  assessment  model 

SFR  -  system  functional  review 

SRR  -  system  requirements  review 

SVR  -  system  verification  review 

TRA  -  technology  readiness  assessment 

TRR  -  test  readiness  review 
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